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SIGNAL DETECTION BASED ON CHANNEL ESTIMATION 

BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention relates to signal processing, and, in particular, to the processing of noisy 
5 received signals, such as high-speed optical signals distorted by linear and non-linear polarization mode 
dispersion (PMD) effects during transmission through an optical transmission path and resulting in data 
pulse broadening and inter-symbol interference (ISI). 
Description of the Related Art 

As transmission speed increases in optical fiber communications, polarization mode dispersion 

10 (PMD) becomes a significant factor limiting system performance, especially for transmission speeds of 
10 Gb/s or higher. PMD causes data pulse broadening and creates inter-symbol interference (ISI). 
Unlike chromatic dispersion which can usually be countered by using a short dispersion-compensated 
fiber, PMD is time varying. While optical solutions have been proposed to counter PMD, they are 
usually very expensive and require a feedback path from the receiver back to the transmitter. 

15 Traditionally, electronic adaptive equalizers have been used to mitigate received signal distortion 

resulting in ISI. Well-known techniques for channels having linear distortion include linear feedforward 
equalization and linear decision feedback equalization (DFE). 

Fig. 1 shows a conventional equalizer 100 used to correct linear signal distortion. Equalizer 100 
has two adaptive equalizers: adaptive equalizer 102 configured to provide feedforward equalization and 

20 adaptive equalizer 104 configured to provide decision feedback equalization. In particular, adaptive 

equalizer 102 receives the current received signal and generates a linearly equalized signal . This 
equalized signal v■^^' is presented to subtraction node 106, which subtracts the feedback signal generated 
by adaptive equalizer 104. The resulting difference signal is presented to slicer 108, which decides 
whether the current received signal Vj^ corresponds to a "1" or a "0" by comparing the difference signal to 

25 a fixed threshold between 0 and 1 (e.g., half way between the reference voltages for logic "1" and logic 
"0" inside the slicer). The selected output data level is fed back to adaptive equalizer 104. In addition, 
the error signal (i.e., the difference between the difference signal generated at subtraction node 106 and 
the sliced signal level generated by slicer 108) is fed back to both adaptive equalizers 102 and 104 and 
used to dynamically control the coefficients within those equalizers using some conventional technique 

30 such as a least mean square (LMS) algorithm. Further information about typical implementations of 
adaptive equalizers 102 and 104 can be found in E.A. Lee and D.G. Messerschmitt, Digital 
Communication^ Kluwer Academic Publisher, 1988, and S.U.H. Qureshi, "Adaptive Equalization," 
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Proceedings of the IEEE, Vol. 73, No. 9, September 1985, the teachings of both of which are 
incorporated herein by reference. 

When the channel is non-linear, such as an optical fiber channel dominated by PMD effects, non- 
linear equalization is used, since the effectiveness of equalizers, such as that shown in Fig. 1, is limited in 
5 those cases. Non-linear equalization techniques range from extremely complex solutions, such as those 
described in U.S. Patent No. 4,213,095, to simple single-tap non-linear DFE modules, such as those 
described in U.S. Patent No. 5,191,462. 

In general, effective non-linear equalization is a very complex and difficult process, involving the 
inversion of the non-linear channel response such that the combined channel and non-linear equalizer 
10 frequency response is flat. The optimization cost functions are often not smooth convex functions and, 
as a result, considerable adaptation convergence difficulties exist. These difficulties are manifested by 
the complexity of the techniques described in U.S. Patent No. 4,213,095. 

SUMMARY OF THE INVENTION 
The present invention is directed to a receiver that relies on channel estimation to mitigate inter- 

15 symbol interference (ISI) problems, especially when the channel is non-linear. When channel non- 
linearity is predominantly quadratic, a channel estimation-based receiver of the present invention 
performs extremely well with much lower complexity than traditional non-linear equalization techniques. 
The adaptive electronic solution of the present invention provides a versatile and cost-effective technique 
for correcting polarization mode dispersion (PMD) problems in optical transmission systems. One 

20 possible application of the present invention is to mitigate fiber PMD in high-speed SONET systems, 
such as OC192, especially where second-order PMD, also known as polarization- induced chromatic 
dispersion (PCD), is strong. 

In one embodiment, the present invention is a receiver for a received signal having two or more 
data levels, the received signal having been transmitted over a transmission channel, the receiver 

25 comprising (a) two or more channel estimators, at least one channel estimator for each different data 

level for the received signal, each channel estimator being configured to model the transmission channel 
to generate an estimated signal corresponding to one of the data levels; and (b) a comparator configured 
to (1) receive the received signal and the estimated signal from each channel estimator and (2) select an 
output data level for the received signal. 

30 In another embodiment, the present invention is a method for processing a received signal having 

two or more data levels, the received signal having been transmitted over a transmission channel, the 
method comprising the steps of (a) generating at least one estimated signal for each data level based on a 
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model of the transmission channel; and (b) processing the received signal and the estimated signal for 
each data level to select an output data level for the received signal. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Other aspects, features, and advantages of the present invention will become more fully apparent 
5 from the following detailed description, the appended claims, and the accompanying drawings in which: 
Fig. I shows a conventional equalizer used to correct linear signal distortion; 
Fig. 2 shows a block diagram of a channel estimation-based receiver, according to one 
embodiment of the present invention, in which the received signal has two possible levels; and 

Fig. 3 shows a block diagram corresponding to a shared-component implementation of the 
1 0 channel estimation-based receiver of Fig. 2. 

DETAILED DESCRIPTION 
Fig. 2 shows a channel estimation-based receiver 200, according to one embodiment of the 
present invention. According to this embodiment, receiver 200 is designed to correct non-linear 
distortion for a received signal v^^ having two different data levels corresponding to logic "1" and logic 
1 5 "0". Receiver 200 has two quadratic (i.e., 2"** order, non-linear) channel estimators: channel estimator 
201 for level "1" and channel estimator 202 for level "0", where each channel estimator adaptively 
generates an estimate for one of the two data levels for the current received signal v^. 

The two estimated signals are present to a comparator comprising two subtraction nodes 215 and 
216 and a compare-and-select module 217. In particular, each estimated signal is subtracted from the 
20 current received signal Vj^ at one of the subtraction nodes and the two resulting difference signals are 
compared at compare-and-select module 217, which decides whether the current received signal 
corresponds to a level "1" or a level "0" (e.g., based on which difference signal has the smallest absolute 
value). 

Level "1" channel estimator 201 of receiver 200 comprises an adaptive equalizer 203 configured 
25 to receive a fixed reference signal v, corresponding to an ideal current level "1" signal. In addition, 

adaptive equalizer 203 receives feedback signals from compare-and-select module 217 corresponding to 
both the previous data output signal(s) as well as the previous error signal(s), where the error signal is 
used to dynamically control the coefficients within adaptive equalizer 203, e.g., using a conventional 
LMS control technique. Adaptive equalizer 203 is preferably implemented based on conventional 
30 adaptive equalizer designs similar to that used for adaptive equalizer 104 of Fig. I, since, in adaptive 
equalizer 203, the tap data are sliced symbols. 
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Adaptive equalizers are typically implemented as finite impulse response (FIR) taps in hardware, 
which consists of multipliers, accumulators, and storage devices to implement equations like 
(ak*coeff(l) + ak.,*coeff(2) + ...), where and aj^., are current and previous inputs. For typical 
feedforward equalizers, where inputs represent either analog or digital input voltages, the required 
resolution is high. For feedback equalizers, inputs are sliced symbols (e.g., 0 or 1 in 2-level cases). 
Since only 1-bit resolution is needed, the multiplier design can be significantly simplified. 

The output signal jc, from adaptive equalizer 203 is processed in parallel through two processing 
paths within channel estimator 201: a order path and a 2"** order path. The l''^ order path multiplies the 
output signal jc, by a order coefficient a, at multiplication node 207, while the 2"** order path squares 
the output signal jCj at squaring node 209 and multiplies the resulting squared signal x^^ by a 2"** order 
coefficient ttj at multiplication node 211. Channel estimator 201 also has a 0* order path, which 
multiplies a fixed reference signal v^f by a 0* order coefficient tto at multiplication node 205. The 
outputs from all three multiplication nodes are summed at summation node 213 to generate an estimated 
level "1" signal v,*^^* corresponding to Equation (1) as follows: 

The estimated level "1" signal ( vf*^^ ) is then subtracted from the current received signal v,^ at subtraction 

node 215 to generate the difference signal for level "1" channel estimator 201 that is input to compare- 
and-select module 217. 

According to the present invention, each of multiplication nodes 205, 207, and 211 are 
dynamically controlled by the error signal generated by compare-and-select module 217 such that the 
coefficients a^, a„ and a2 are all adaptive coefficients that are dynamically adjusted, e.g., using an LMS 
control technique. (Note that, in Fig. 2, the thick arrow is used to signify that the error signal is input to 
each of multiplication nodes 205, 207, and 211.) In preferred implementations, the P* order coefficient 
a, has a fixed value of 1. As such, the order path can be implemented simply as a signal path 
connecting the output of adaptive equalizer 203 directly to the corresponding input of summation node 
213 without requiring multiplication node 207. 

Level "0" channel estimator 202 is analogous to level "1" channel estimator 201, with elements 
204, 206, 208, 210, 212, and 214 of level "0" channel estimator 202 configured to operate in analogous 
fashion to elements 203, 205, 207, 209, 211, and 213 of level "1" channel estimator 201, to generate the 
estimated level "0" signal, which is subtracted from the current received signal Vi„ at subtraction node 
216 to generate the difference signal for level "0" channel estimator 202 that is input to compare-and- 
select module 217. Note that the input to level "0" channel estimator 202 is a fixed, level "0" reference 
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signal Vq, which is different from the fixed, level "1" reference signal v,. The resulting estimated level 
"0" signal generated by channel estimator 202 can be represented by 

Vo"'=)ffo+A^o+A^o (2) 

where Xq is the output signal from adaptive equalizer 204 and Pq, Pi, and P2 are the adaptive 0*, V\ and 
5 2"** order coefficients applied at muhiplication nodes 206, 208, and 212, respectively, and dynamically 
controlled based on the error signal generated by compare-and-select module 217. 

By including the 2""* order terms ( ^2X1^ and >ff2^o )' channel estimators 201 and 202 model a 

general 2"** order system. If the transmission channel for received signal v^^ is dominated by 1^* and 2^"^ 
order effects, then channel estimators 201 and 202 should produce accurate replicas of the two possible 
10 channel outputs after they have been properly adapted to model the channel The present invention 

avoids the typically difficult problem associated with conventional non-linear equalization schemes of 
having to invert the channel response. 

Fig. 3 shows a block diagram corresponding to a preferred shared-component implementation of 
the channel estimation-based receiver 200 of Fig. 2. According to this shared-component 
1 5 implementation, receiver 300 is implemented with only a single adaptive equalizer 302, which differs 
i n from adaptive equalizers 203 and 204 (e.g., the first tap corresponding to the current received signal is 

stripped). The output signal from adaptive equalizer 302 is used to generate the input signals for both 
a channel estimators 201 and 202. 

In particular, the level "1" reference signal is multiplied by a coefficient c, (e.g., corresponding 
\M 20 to the first tap of a conventional adaptive equalizer) at multiplication node 303 and the resulting product 
(Vi*c,) is added to the output signal from adaptive equalizer 302 at summation node 305 to generate the 
input signal to channel estimator 201. Analogously, the level "0" reference signal Vo is multiplied by the 
coefficient c, at multiplication node 304 and the resulting product (Vo*c,) is added to the output signal 
from adaptive equalizer 302 at summation node 306 to generate the input signal to channel estimator 202. 
25 Although the two channel estimators (e.g., 201 and 202 of Figs. 2 and 3) are shown with different 

sets of coefficients (tto, 0£„ a2) and (Po, Pi, P2), in some implementations, the different channel estimators 
could be implemented using a single set of coefficients. In that case, the potential exists for even fiirther 
sharing of components, where two or more channel estimators temporally share the same set of 0*, 1^, 
and 2"** order path components, with different inputs being applied at different times corresponding to 
30 different channel estimators. Such temporal sharing would add latency to and decrease the speed (i.e., 
data throughput) of the receiver, which could make such an implementation unacceptable for certain 
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applications. Another way of sharing is to share the LMS update circuit for the coefficients and just copy 
the updated results for the two channel estimators. 

Channel estimation-based receivers 200 and 300 may be implemented in software or in hardware, 
in either the digital domain or the analog domain, or in combinations thereof For high-speed 
applications (i.e., data rates as high as 10 Gb/s or even higher), the receivers are preferably implemented 
in hardware in the analog domain. 

Channel estimation-based receivers 200 and 300 are designed with the following characteristics: 
o The received signal v^^, has only two data levels ("1" and "0"); 
o The channel estimators contain only 0^ 1'*, and 2"** order terms; and 
o Only the current and past data signals are fed into the adaptive equalizer(s). 
Those skilled in the art will appreciate that the present invention can be implemented for applications 
having other characteristics. 

For example, channel estimation-based receivers of the present invention can be implemented for 
applications in which the received signal has more than two data levels. In that case, the receiver is 
implemented with more than two channel estimators: (at least) one channel estimator for each different 
data level, where the estimated signal generated by each channel estimator is subtracted from the current 
received signal to generate a difference signal for the compare-and-select module, which still preferably 
selects the output data level for the current received signal based on the difference signal having the 
smallest absolute value. 

Furthermore, channel estimation-based receivers of the present invention can be implemented for 
applications in which the channel estimators contain one or more additional, higher-order terms beyond 
the 2"** order term. In that case, each channel estimator will have an additional processing path for each 
additional, higher-order term comprising, for example, a sequence of multiplication nodes to generate the 
higher-order term, where one of the multiplication nodes is an adaptive node corresponding to a 
dynamically controlled coefficient for the higher-order term. 

Moreover, channel estimation-based receivers of the present invention can be implemented for 
applications in which each adaptive equalizer receives future data as well as the past and current data as 
tap inputs. In that case, the receiver should be implemented with a different channel estimator for each 
different combination of current and future data. For example, for a receiver designed for a received 
signal having two data levels "1" and "0" where the adaptive equalizers take into account the current data 
value and one future data value, the receiver will preferably have four different channel estimators: one 
for each different combination of possible current and future data values (i.e., 00, 01, 10, and 1 1). If the 
adaptive equalizers are configured to take into account two future data values, then the receiver will 
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preferably have eight different channel estimators corresponding to the eight different combinations of 
one current and two future data values. 

These different characteristics can be combined such that, in general, a channel estimation-based 
receiver according to the present invention will preferably have w^' channel estimators: one channel 
5 estimator for each different possible combination of the current data value and n future data values, 

where each data value has m different possible levels, where m is an integer greater than 1 and « is a non- 
negative integer, and each channel estimator has p+\ processing paths corresponding to (0*, 1'*, 2"**, 

order terms, where p is an integer greater than 1. In preferred shared-component implementations, 
different channel estimators will share adaptive equalizers. In one possible shared-component 
10 implementation, all w^* channel estimators will share a single adaptive equalizer with suitable 

components provided to generate the appropriate input signal for each different channel estimator from 
the single output signal generated by the shared adaptive equalizer. 

Although the present invention has been described in the context of processing electrical signals 
corresponding to optical signals transmitted over optical fiber transmission channels, the present 
1 5 invention can also be applied to signals received from other types of transmission channels, including 
electrical or wireless (i.e., "over-the-air") transmission channels. 

The present invention may be implemented as circuit-based processes, including possible 
implementation on a single integrated circuit. As would be apparent to one skilled in the art, various 
functions of circuit elements may also be implemented as processing steps in a software program. Such 
20 software may be employed in, for example, a digital signal processor, micro-controller, or general- 
purpose computer. 

The present invention can be embodied in the form of methods and apparatuses for practicing 
those methods. The present invention can also be embodied in the form of program code embodied in 
tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other machine-readable storage 

25 medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, 
the machine becomes an apparatus for practicing the invention. The present invention can also be 
embodied in the form of program code, for example, whether stored in a storage medium, loaded into 
and/or executed by a machine, or transmitted over some transmission medium or carrier, such as over 
electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the 

30 program code is loaded into and executed by a machine, such as a computer, the machine becomes an 

apparatus for practicing the invention. When implemented on a general-purpose processor, the program 
code segments combine with the processor to provide a unique device that operates analogously to 
specific logic circuits. 
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It will be further understood that various changes in the details, materials, and arrangements of 
the parts which have been described and illustrated in order to explain the nature of this invention may be 
made by those skilled in the art without departing from the scope of the invention as expressed in the 
following claims. 
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